Hi, I’m Matt. One of the unique aspects of Ollama is its dual nature as both an end-user tool and a developer tool with an API endpoint. This allows developers to program against it. In fact, the GitHub repo homepage lists numerous community integrations, including desktop and web tools - over 30, 40, or even 50 entries! With so many options, it can be overwhelming to choose the right one for your needs.

To help with this decision-making process, I started creating a chart and document on my blog at technovangelist.com that highlights the differences between these various tools. However, as I began exploring the first few integrations, I discovered that many of them had significant issues. It felt like I just highlighting the bad stuff rather than trying to be helpful.

And so I decided to create a video series that looks at each of these tools. My intention is to provide helpful insights into each app, highlighting what makes them compelling and point out what might be missing from their offerings, from my perspective. Perhaps this will help you find the perfect tool for your needs or even inspire you to build your own application that combines the best features of multiple tools. Let’s take a closer look at some of these desktop and web apps and see what they have to offer!

It’s important to remember that while I was a founding member of the Ollama team, I am no longer working for Ollama. So these are simply my opinions and not reflective of anything that anyone in the Ollama team believes.
this is a test
Okay, so let’s look at one of these tools. With so many it’s hard to know where to start, so I decided to choose the application that I think is among the favorites for most people that are working with Ollama. It’s the application that I see most often mentioned in the Ollama Discord, which by the way, you can find at discord.gg/ollama. And it’s called OpenWebUI. It used to go by something like Ollama WebUI, but I think the author wants to be less tied to Ollama specifically.

There are a lot of things to really like about this tool. It is visually very attractive to look at. It has a few themes, so you can tweak it to be the way you prefer, whether it’s light mode or dark mode, and I wish dark mode would die…it hurts my eyes. It gives me good suggestions about some things I could try, although I wish those suggestions would change occasionally. And it also has a very opinionated way of working with prompts, which is interesting. I initially thought this was a little distracting but I think I came around to love it. Let’s take a look.

So when you first bring up the tool, you are asked to login. I hate it when tools do this. If something is running locally, don’t make me sign in. If you want to provide this layer as an option for those who want to host it, great, but make it an option.

In this UI, we have a few sections. Front and center is the chat window. At the bottom we can enter in a message and press enter and then see the results in the main window. Right at the top we can choose a model. This is a pretty critical step so I wish it wasn’t as visually hidden. On the left we see a big button for a new chat, then curiously sections for modelfiles and prompts, I’ll come back to those, and then documents. Then below that is a list of chats you have had along with a search box for searching those chats.

Let’s ask a question, why is the sky blue. This is my go to question. And my model is set to phi3 which is a tiny model that is performing beautifully for so many use cases. Pretty quickly we get a pretty amazing response.

But I’m already getting ahead of myself a little bit. Let’s go to the webpage and try to figure out how to get this thing installed. We can go to ollama.com and then click on the GitHub link. Now this is the repo readme. Scroll all the way down to the end. Well, almost all the way down to the end. And we get to the top of the community integrations list and look at the web and desktop tools. And here is OpenWebUI at the top.

This takes us to the Github repo for open webui. Looking at the readme, it looks like an incredible tool. I see local rag integration, intuitive interface, code syntax highlighting, theme customization, markdown and latex support, web browsing capability, and so much more. Looks really really cool. And this list of features goes on and on. Let’s scroll up closer to the top, and click on the releases link. And this is interesting because we can see that it’s not just one release. There was a release just in the last couple of days. And we can scroll through here and we can see that there are releases happening on a pretty regular basis. And they’re not small releases, they’re not just documentation fixes. They’re significant new features that are being added, which is incredible.

So let’s look at the installation process. Near the top is how to install. And right below that is a quick start with Docker. Now not everyone loves Docker. Some people don’t like it for whatever reason. They might have misunderstood the licensing kerfuffle that happened a few months or a few years ago, thinking that they need to pay for Docker when most individual users don’t need to pay anything.

But if you are working with Docker, and Ollama is running locally, then you can use this command. -d says run this as a daemon process with the UI running in the background. -p publishs the port 8080 in the container to port 3000 on your host. If you would prefer to access it on a different port, replace 3000 with anything else. the add-host parameter says to use ollama on the local machine. -v creates a volume located at open-webui and mounts that to /app/backend/data in the container. name sets the name of the container and “restart always” means that if it stops for any reason, its coming back up. And finally we are specifying that the container registry and image we should use is on ghcr.io.

If you don’t want to use docker, there are instructions you can follow over at this link.

But once you have it running, You should be able to visit localhost and the port, which for me is 3000. And here we have that sign in page. The first time we get here we need to create a new account so click sign up and enter you name, an email and password and click create. Now you are dropped into the ui.

The first thing you probably want to do is set the model to use. Come up here to this dropdown and type in the name of a model you want. I have a lot of models available to me because I have downloaded and created so many. Here I have chosen llama3. and now you can add a question.

lets go to the settings, which you can find up top here or down in the bottom corner at your name. Under general we can set a theme and language. But then there is system prompt. I’m not sure why anyone would want to set, what I assume to be a global system prompt. Does this get overriden on most models, because the system prompt is set in the model. And there is no way to find out without digging around some docs. And then it looks like we can set some more default parameters which you really want to be different for each model…

Next is connections. we can set an openai api url as well as an api key. or update the url to use for ollama. And I can also add additional hosts. Not sure what happens when I have multiple hosts because I can’t choose the host for a model when i set the model to use. So that’s strange.

For models we can set the host to use to manage models. Its not clear if this is different than the connections screen, and even though I set a second host, i only see the one default host here. To the right is a button to update all models. Unless you only have a small number of models, do not click this button. It updates models in the most inefficient way possible, pulling every model in your list. It hung on my machine. Then you can pull a new model if you know the name of it, though this is a much more awkward way than the cli. You can also delete models from here.

It’s interesting that they offer a way to upload what they call models as the source gguf files, but those are just the model weights and not the full ollama model, so they are a bit useless until you make the model from that. And then you can do some management of models that you access through litellm.

In interface we have some functionality options and some default prompt suggestions. I’m not sure what the value of this is. I could imagine some default system prompts would be a lot better served here.

In audio we have the ability to set a speech to text engine, either using whisper locally or using open ai. but we have no control over which whisper models to use or which language you speak. And then there is text to speech and again there is local or openai. The local one is all but useless.

for images we can use this to access automatic1111 which is a bit strange, but the author seems to want to support more than just ollama, so that seems useful to some. And that’s all that is really interesting here.

so far everything feels pretty good about open web ui. Things work roughly the way you would expect and it generally seems to perform well. But here is where we start getting a bit more interesting.

lets come up to the place we can choose a model. There is the dropdown but there is also this plus sign. So if i click this I can add a second model, and a third, and fourth, and so on. So what happens now? Well, ask a question and you shall see. Why is the sky blue. We wait for it to respond, and we can read the answer. Take a look at the top. I set it to use phi3 and llama3 and so now at the bottom I see it showing 2 of 2. Click the arrow. Its now showing me the output of the other model. So now you have a way of configuring multiple models and its going to ask the same question to each. There is no magic here, its going to take longer to ask more models, but it becomes a bit easier to find the right model for you.

There is also a thumbs up and thumbs down button here. The readme refers to some ability to use this to make something better, but there is no info on where the data is going. I couldn’t find any way to get the info out. I looked in the SQLite db the app creates and nothing seems to have been added. You can also edit the answer. Again maybe this is to help come up with a series of good questions and answers to help fine tune later, but the feature doesn’t seem to be complete.

Ok, lets go up to prompts on the top left corner and click the plus sign to add a new prompt. here you can add a phrase you want to use as the beginning of your prompt. Let’s try summarize. enter summarize for title and it will default to using the same thing for command. And then enter the prompt. "Write a short summary of the following text: ". Click Save and Update.

Now go to a new chat and type the slash key. press enter to select summarize. Go grab some text from somewhere and paste it in and press enter. And we have a summary. But we had to go through a few extra steps there. So go back to that prompt, and add the word CLIPBOARD in double curly braces. Save and update and lets go back to a chat. The text is still in our clipboard, so type slash summarize and enter. It has automatically taken what’s in my clipboard and added it to the end of the prompt. That’s awesome. But maybe you want to summarize it differently depending on the age of the person you are going to share this with. Add ‘for someone who is’ and then age in square brackets ‘years old’. Go to the chat and bring that up again. the text gets pasted in, and age is highlighted, waiting for you to enter an age. You can add as many ‘variables’ in square brackets as you like. And as many prompts as you like.

On that prompts page, there is also a link to discover a prompt, but it just takes you to the open webui webpage. You can scroll down to find a list of prompts folks have shared, but it really should link you right to the right place. Even so I didn’t find that all that useful. I have found the prompts tend to be super generic and its better to come up with your own. that said, the whole slash command to bring up prompts is amazing and it may be my favorite feature of the app.

let’s go back to the chat. Now press the pound key, or you may call it the hash key. type https://technovangelist.com. Now you can ask any question about content on that page. It doesn’t seem to follow any links though, its just what’s on that page. So I can ask about my patreon, but asking about my employment status with ollama results in a very generic answer.

back up on the top left is a section called documents. click on it and then the plus sign to add a document. You can add one document or a bunch. Unfortunately you won’t get any feedback until the process is done. But once the doc shows up, you can also add tags to the document. Now you can come back to the chat and type the hash key again and you will be prompted to choose a document, or a collection which is the tags associated with your docs. It seems to do a pretty good job here. I wish there was some feedback but maybe that will come soon.

There are some document settings, but they don’t seem to make any impact. I have changed them and the next time I go back to this page, they are changed back to the default. Maybe this is something that needs fixing.

The last thing we haven’t looked as is the Modelfiles section. I have no idea why this exists. they should just make a better experience working with models on the ollama site because it feels like this is just trying to do the same thing but just more awkwardly. I wish this would make it easier to take any model, then apply new parameters and system prompt, give the model a new name and then save it back to the ollama library. The CLI already does that but some folks prefer a UI so would be a nice addition. Also having another key like backslash or pipe that lets me change the model used would be cool too.

I think I have covered everything. So what do I think overall? It’s a nice tool but some of the weird aspects are hard to get past. the whole user thing is a pain though I am sure they worked hard on that. I wish you could just disable that entirely. What they are doing with the modelfile section doesn’t make any sense, but having a better workflow with models would be great. And that button to update models is way to easy to press so either make it harder to press, or update it to compare the hashes first and then do the pull. The slash command to pull up prompts is pretty genius. I love it. But that’s the only differentiating feature from the CLI. Is that enough to use it more regularly? I don’t know. I know there are a lot of folks who are scared of the command line and will be ok with it being a 1 to 1 version of the cli tool, but that’s not me.

I thought this would be a short video when i started putting it together, and I didn’t expect to like it as much as I did. I hope this will be the experience with the other tools on the community integrations list.

What do you think? Do you use Open WebUI? Is there anything I missed? Do you think there is a better tool out there? Share that with me in the comments below. I love what folks share in the comments. Be sure to subscribe to my newsletter. You can find it at technovangelist.com/newletter.

Thanks so much for being here…goodbye